The present disclosure relates to image processing apparatuses and methods, and more specifically, to an image processing apparatus and method of generating a wide-view image considering characteristics of an input image.
A flat panel display has recently been increasing in size. When a visual range is fixed, a larger flat panel display leads to a better sense of realism (immersive experience) with viewing of wide-view images. In this Specification, the expression of “a sense of realism” means a viewer's feeling like actually being in the world represented by images on the display.
A display with a high resolution such as 4K and 8K is now about to be available for practical use. The higher resolution of a display allows representation of images that look close to real, and leads to better realistic effects. In this Specification, the expression of “realistic effects” means a viewer's feeling as if the object of an image he/she is looking actually exists.
For displaying images on a high-resolution large-sized display, the images are often displayed with a resolution lower than that of the display, e.g., when the resolution of a display is 4K, images are often displayed with a resolution of 2K.
If this is the case, the resolution of the display images is up-converted to that of the display using linear scaling techniques adopted in the display, e.g., superresolution techniques. The images are then displayed as wide-view images.
The concern here is that when an image is scaled up uniformly over the screen by the linear scaling techniques, the viewability is impaired because the image in the viewer's central field of view (discriminating field of view, useful field of view) looks different from that originally intended at the time of imaging. The details will be described later.
Therefore, image up-conversion using nonlinear scaling techniques is proposed to change the scale factor between the central part of the screen that is very likely to include any attention-attracting object, and the remaining part of the screen peripheral to the central part (hereinafter, referred to as peripheral part). In this Specification, the expression of “viewability” means the easy-to-view degree of the images, and the easy-to-understand degree of the contents of the images.
The nonlinear scaling techniques include a technique of performing scaling on the central part of the screen and the peripheral part thereof using various scale factors. With this technique, the scale factors are defined differently for the central part of the screen and the peripheral part thereof based on a visual range, for example. As an example, refer to Japanese Patent Application Laid-open No. 2007-264456. There is another technique of performing scaling on the central part of the screen and the peripheral part thereof using various scale factors. With this technique, the scale factors are also defined differently for the central part of the screen and the peripheral part thereof but with this technique based on the motion in the entire screen. As an example, refer to Japanese Patent Application Laid-open No. 2008-242048.
There is still another technique of performing scaling on the central part of the screen and the peripheral part thereof using various scale factors. With this technique, a viewer's line of sight is detected by the movement of his eyeballs, and based on this detected line of sight, the scale factors are defined differently for the central part of the screen and the peripheral part thereof. As an example, refer to Japanese Patent Application Laid-open No. 2008-233765. There is also a technique utilizing Seam Carving. As an example, refer to “Seam Carving for Content-Aware Image Resizing”, Avidan et al, SIGGRAPH 2007.
These non-linear scaling techniques, however, may cause image deformation due to the different scale factors between the central part of the screen and the peripheral part thereof.
In order to prevent such image deformation, proposed is a non-linear scaling technique with which an input image is disposed in the central part of the screen, and another image is disposed by extrapolation in the part peripheral to the central part. As an example, refer to “Multi-scale ultrawide foveated video extrapolation”, A. Adies, T. Avraham, and Y. Schechner. Israel Institute of Technology In ICCP, 2011.
In order to prevent image deformation, there is another non-linear scaling technique with which an input image is disposed in the central part of the screen after resizing and/or position change thereof. In the part peripheral to the central part, an image is disposed by extrapolation around the central part. As an example, refer to Japanese Patent Application Laid-open No. 2009-162899.